A Brief Index for Proximity Searching

Many pattern recognition tasks can modeled as proximity searching. From nearest neighbor classification to multimedia databases the common task is to quickly find all the elements close to a given query. This task can be accomplished very easily by sequentially examining all the elements in the collection, but turns to be impractical in two situations: when the distance used to compare elements is expensive or when the numbers of elements is very large (in the order of billions of objects). Recently an improvement over previous approaches has been done by using permutations instead of distances to predict proximity. Every object in the database record how the set of reference objects (the permutants) is seen, i.e. only the relative positions are used. When a query arrives the relative displacements in the permutants between the query and a particular object is measured. The permutation of every object is represented with κ short integers in practice, producing bulky indexes of size κn. In this paper we show how to represent the permutation as a binary vector, using just one bit for each permutant (instead of log κ in the plain representation). The Hamming distance in the binary signature is used then to predict proximity between objects in the database. We tested this approach with many real life metric databases obtaining a recall close to the Spearman ρ using 16 times less space.